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Virtual reality 


Introducing . . . 



sensorama 


The Revolutionary Motion Picture System 
that takes you into another world 
with 


WIDE VISION 


COLOR 


STEREO-SOUND 


AROMAS 
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TASCAR - virtual acoustics 
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TASCAR 


Toolbox for Acoustic Scene Creation And Rendering 


■ Linux Software 

■ Acoustic Environments 

■ Scenes Design 


■ Real Time Rendering 

■ Moving sources, Acoustic modelling 

■ Interactive 
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TASCAR - origin 
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■ Initial idea: Music Performance (Harmony of the Spheres) 

■ Inspiration: Sound Scape Renderer 

■ Further development in Research Project 
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Evolving hearing aids 
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■ Much more than simple amplification 

■ Space-aware signal processors 

■ Noise and feedback cancellation 

■ Directivity patterns, source localisation 

■ New methods for evaluating spatial properties are needed 
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Example : how directionality in hearing aid improves the speech 
intelligibility? 



■ Typical test set up 

■ Two separate sources 

■ For some directions noise 
can be fully suppressed 

■ Result: High gain from using 
directional hearing aid. 

■ Oversimplification 

■ Subject behaviour is 
unnatural 

■ Unrealistic 
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Example : how directionality in hearing aid improves the speech 
intelligibility? 



■ simulation of an acoustic 
environment 

■ all channels active - 
reflections, reverberation 

■ much more realistic 

■ moving sources 

■ Result: Benefit from using 
directional hearing aid is not 
that big anymore 

■ Results correspond to the 
real-life situation 
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■ Realistic, but... 

■ not scalable 

■ uncontrollable 

■ costs time and effort 
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conditions! 

a Need: 
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a Need: 

■ complex, realistic 

m controllable and reproducible test environments 

■ Plausible reproduction for human listeners 

■ Physically correct simulation for listening with a hearing aid 

■ Solution: 

■ create a virtual acoustic environment using TASCAR 

■ play back using loudspeaker array 

Bring the real life into the lab! 
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TASCAR: How does it work? 
Modular structure 
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Audio content 


sound files or real-time audio 


Geometry 


dynamic objects, image source positions 


Acoustic model 


transmission and reflection model, rendering to output format 
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Built upon jack audio connection kit: 


Audio content 


sound files or real-time audio 




Geometry 


dynamic objects, image source positions 


c 


Acoustic model 


transmission and reflection model, rendering to output format 
jack ports jack time line 
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Acoustic model: 
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■ Built from one or more “sound vertices” 

■ Each sound vertex is omni-directional 
radiator 

■ Each sound vertex is a jack input port 




9.4.2015 TASCAR 


13/28 









Acoustic model: 
Receiver 
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■ Encode direction into audio signal 

■ jack output ports 

■ Various output formats: 

■ omni and cardioid virtual microphones 

■ 3rd order ambisonics, horizontal and 
full-periphonic 

■ Horizontal ambisonic panning, arbitrary 
order, ’basic’ and ’in-phase’ 

■ Binaural rendering with virtual loudspeakers 
and head-related transfer functions 

■ Reproduction methods validated with 
different classes of hearing aid algorithms. 
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Acoustic model: 
Image source model 
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i 


m Polygon-mesh reflectors, dynamic location 
and orientation 

■ Surface properties approximated by 1st 
order low-pass filter 

■ Time-domain processing: 

no pre-calculation of impulse responses 

■ Soft fades at reflector boundaries: 


/ 

>^receiyer 

3 primary source 

reflector \ 

\ 

\ 

\ 

\ 

\ 
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} image source 
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■ Surface properties approximated by 1st 
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Acoustic model: 
Transmission model 
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■ Time variant delay line 

■ Delay line for each image source 

■ Doppler shift for fast movements 

■ Time varying comb filter effects for slow 
movements 

■ Air absorption model 

■ Simplification: 1st order low-pass with 
distance-dependent filter coefficients 

■ Good match up to 100 m 

■ 18°C, 60% rel. humidity 
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Acoustic model: 
Boundary volume masks 
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■ Allow to split large scenes into sub-spaces 

■ Zero-gain if receiver is within (“inner” masks) 
or outside (“outer” masks) masks 

■ Soft boundaries, von-Hann ramp 
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Acoustic model: 

Room coupling simulation 
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■ Sound portals = dedicated sound sources 

■ Input: output of microphone in adjacent room 

■ Rectangular area, always radiating from 
closest point to receiver 

■ Increased effective distance to avoid 
proximity effect 

X receiver 


sourjd portal 
CD eff. source 
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Diffuse source model 
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■ First order ambisonics (FOA) recordings for 
diffuse background sounds 

■ Diffuse reverberation in first order 
ambisonics with external tools: 

■ Convolution reverb 

■ Feedback delay networks 

■ No transmission model: 

■ Constant gain 

■ No Doppler shift 

■ Requires FOA decoder for each receiver 
type 
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■ Trajectories 

defined by sampled location and orientation 

■ Delta transforms 

controlled via OSC or geometry modifier plugins 

■ Updated on each processing cycle 

■ Derived parameters interpolated within processing blocks 

■ All objects can be dynamic: 

sources, receivers, reflectors, volume masks 
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Tradeoffs, bugs and limitations 
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TASCAR is not a room acoustic simulator 


Simplifications of reflections and sound propagation: 
Missing/simplified diffraction model 
Missing absorbers and obstacles 

Real-time processing limits complexity 
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TASCAR is not a room acoustic simulator 
Motion blurs and displaces sources 


Linear interpolation of panning coefficients, e.g., 

ambisonics weights, within blocks 

Acoustic travel time not taken into account in trajectory 

processing 
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Tradeoffs, bugs and limitations 
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TASCAR is not a room acoustic simulator 
Motion blurs and displaces sources 
Acoustic artifacts 


Delay lines use nearest neighbor interpolation 

Reproduction methods may not create correct physical 
sound field 
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M Tradeoffs, bugs and limitations 
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TASCAR is not a room acoustic simulator 
Motion blurs and displaces sources 
Acoustic artifacts 
Huge code base 


Transparent system: 

■ Impact on research applications is continuously assessed. 

■ Awareness of limitations is shared with users. 
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■ Interactive virtual acoustic environment 

■ Coupling to computer graphics (blender game engine): 

■ Game logic processing: 

blender ->■ TASCAR (e.g., receiver position) 

■ Visual representation: 

TASCAR -» blender (e.g., generated positions) 

■ Demonstrating all described aspects of TASCAR 
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pendulum music 


harmony of the spheres 
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jazz club 


of the spheres 


jazz club 



Explore the stage 
and room acoustics 



Dynamic feedback 
loop simulation 


harmony of the 
spheres 


Doppler shift and 
distance perception 
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■ Today 15:30 (Installation space): 

Hands-on workshop, 8 channel loudspeaker reproduction 

■ Saturday 11:30 (poster session, Foyer): 

Play and learn, headphone system 

■ Any other time: 

Ask Joanna or Giso 
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■ Universitat Oldenburg: 

Core development 

Open source; undocumented but functional 

https://github.com/gisogrimm/tascar 

■ HorTech gGmbH: 

Maintenance and support 

User interfaces, data logging, customization 

Commercial license or project participation 

http://www.hoertech.de/ 
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Conclusions 
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■ Tool for rendering dynamic spatially complex acoustic 
environments 

■ Include movement of sources and listeners 

■ Full control, scalable and reproducible 

■ Focus on plausibility and basic acoustic properties for 
hearing aid research 
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■ Focus on plausibility and basic acoustic properties for 
hearing aid research 


Outlook 


Diffraction model, directional sources 

Many more TASCAR-based studies in hearing aid research 

Large VR lab in Oldenburg, powered by TASCAR 
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Thank you for your attention! 
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Examples of research applications 
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#1: Space-aware hearing aids and head motion 
(PhD project Maartje Hendrikse) 

#2: The Influence of Dynamic Binaural Cues on Speech 
Intelligibility 

(Diploma thesis Jan Heeren) 

#3: Spatial dynamics and postural stability 
(Bachelor thesis Isabel Busing) 


9.4.2015 TASCAR 


29/28 









Example 1: 

Space-aware hearing aids and head motion 
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■ Can we identify individual head movement strategies? 

■ Are individual head movement strategies task dependent? 

■ Will hearing aid performance be affected? 

■ Apparatus: 

■ 8 channel loudspeaker array 

■ 9 DOF wireless head motion sensor 

■ ’kinect’ depth camera (skeleton tracker) 

■ Electro-oculography (gaze direction) 


Work in progress, preliminary data. 
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Example 1: 

Space-aware hearing aids and head motion 
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Panel discussion 
Early reflections 
Task: 

Carefully listen to the 
discussion. 

Recording of head 
orientation 
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Example 1: 

Space-aware hearing aids and head motion 
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Example 2: “The Influence of Dynamic Bin¬ 
aural Cues on Speech Intelligibility” 
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■ Binaurally ambiguous 
listening positions - 

Small binaural cues like 
ITD or ILD 

■ Assumption: Dynamic 
cues e.g. minimal head or 
source movements 
contribute to a better spatial 
resolution in complex 
listening situations. 

■ Better localisation 

■ Better speech 
intelligibility 



■ What is the effect of the 
dynamic binaural cues ? 
- measuring SRT in 
dynamic conditions 
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Example 2: “The Influence of Dynamic Bin¬ 
aural Cues on Speech Intelligibility” 
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Example 2: “The Influence of Dynamic Bin¬ 
aural Cues on Speech Intelligibility” 
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Hypotheses proved: 
m Speech intelligibility 
increases if sources 
move towards different 
ears(Rot by S0N180 and 
Cou by S0N0) 

■ Speech intelligibility 
decreases if sources 
move towards the same 
ear (Cou by SON 180 and 
Rot by SON 180) 



SONO SON180 SONO S0N180 
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Example 3: 

Spatial dynamics and postural stability 
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Context Some HA users feel disturbed by fast-acting automatics 
Literature Static acoustic information supports sense of balance 
Goal Explore impact of spatially dynamic acoustic information 
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Context Some HA users feel disturbed by fast-acting automatics 
Literature Static acoustic information supports sense of balance 
Goal Explore impact of spatially dynamic acoustic information 


Test conditions 


10 young participants with normal vision and hearing 
Task (clinical): “Fukuda Stepping Test” 

■ Test setup: TASCAR with Kinect camera 
Three control parameters: 

Complexity Dynamics Vision 
simple static eyes open 
complex dynamic eyes closed 
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Example 3: 

Spatial dynamics and postural stability 
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